feat: add explicit ToplingDB runtime selection by imbajin · Pull Request #3275 · apache/hugegraph

imbajin · 2026-10-06T08:54:28Z

Purpose of the PR

Make ToplingDB an optional storage engine while preserving HugeGraph APIs. Standard RocksDB remains the default.

How to switch

Run bash bin/prepare-topling.sh following the setup guide.
Set rocksdb.provider=topling in the standalone graph properties and export TOPLINGDB_ROCKSDB_PROVIDER=topling.
Initialize the fresh graph with bash bin/init-store.sh, then run bash bin/start-hugegraph.sh.

PD and Store use their own configuration and start scripts. ToplingDB offers performance-focused storage and flexible configuration.

Original preparation reference

Main Changes

Prepare and select a component-local ToplingDB runtime before startup.
Preserve cache lifetime and release request, native and service resources.

Verifying these changes

Trivial rework / code cleanup without any test coverage. (No Need)
Covered by existing tests and focused regression tests.
Java 17 compilation and installation with Maven 3.10; relevant runtime and lifecycle checks.

Current CI results are shown in the checks below.

Does this PR potentially affect the following parts?

Documentation Status

Doc - TODO
Doc - Done: repository guide and website PR #510; coordinate both merges.
Doc - No Need

codecov · 2026-10-06T09:00:31Z

Codecov Report

❌ Patch coverage is 57.09970% with 142 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.29%. Comparing base (af3c686) to head (96f99de).

Files with missing lines	Patch %	Lines
...a/org/apache/hugegraph/store/HeartbeatService.java	55.73%	23 Missing and 4 partials ⚠️
...gegraph/backend/store/rocksdb/RocksDBSessions.java	4.54%	21 Missing ⚠️
.../hugegraph/backend/store/rocksdb/RocksDBStore.java	36.36%	20 Missing and 1 partial ⚠️
...he/hugegraph/backend/tx/GraphIndexTransaction.java	45.00%	7 Missing and 4 partials ⚠️
...a/org/apache/hugegraph/pd/TaskScheduleService.java	0.00%	9 Missing ⚠️
.../org/apache/hugegraph/pd/meta/MetadataFactory.java	58.82%	6 Missing and 1 partial ⚠️
...rc/main/java/org/apache/hugegraph/HugeFactory.java	61.11%	6 Missing and 1 partial ⚠️
...graph/backend/cache/CachedSchemaTransactionV2.java	77.41%	2 Missing and 5 partials ⚠️
...ugegraph/backend/cache/CachedGraphTransaction.java	70.00%	3 Missing and 3 partials ⚠️
...raph/backend/store/rocksdb/RocksDBStdSessions.java	54.54%	5 Missing ⚠️
... and 9 more

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3275      +/-   ##
============================================
+ Coverage     40.46%   41.29%   +0.83%     
- Complexity     6566     6815     +249     
============================================
  Files           751      766      +15     
  Lines         63982    66317    +2335     
  Branches       8502     8806     +304     
============================================
+ Hits          25889    27386    +1497     
- Misses        35175    35857     +682     
- Partials       2918     3074     +156

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

bitflicker64

Blocking: yes. Summary: The new current-schema lookup in GraphIndexTransaction throws for vertices and edges whose label was deleted. Those left records can no longer be dropped, and a normal vertex with a left edge can no longer be dropped either. Both deletes worked before this PR. I found nothing else actionable in the launcher selection, the RocksDB provider check, the Topling truncate path, the cache lifetime change or the PD/Store shutdown rework. CI on this head is red, and both failures look unrelated: RateLimiterTest timing in server_memory, and dependency-review flagging the existing netty-handler 4.1.52 in hg-pd-dist. Evidence: static trace of StandardHugeGraph.vertexLabelOrNone/edgeLabelOrNone -> VertexLabel/EdgeLabel.undefined(graph, id); SchemaTransaction.getVertexLabel/getEdgeLabel return null for a missing id; GraphTransaction.removeVertex and prepareDeletions -> updateVertexIndex/updateEdgeIndex -> GraphIndexTransaction.indexLabelIds -> currentSchemaLabel E.checkArgument; GraphTransaction.warnLeftRecord shows these records are returned to callers.

- align Netty, gRPC and protobuf runtime versions - preserve module dependency and test configuration - include packaged dependencies and matching licenses

- add explicit provider selection and compatibility checks - preserve standard RocksDB as the default - cover runtime selection and utility scanning

- verify the JNI package and EasyMigrate capability - install readable runtime assets outside standard libraries - provide shared preload selection and native configuration

- select one runtime for Server, init and dump commands - keep inherited configuration scoped to the selected component - cover launcher selection and upgrade contracts

- close session and database native resources reliably - preserve commits and clear reusable write batches - cover native session disposal and reuse

- retain shared caches and invalidation listeners across requests - dispose graph-scoped holders when the graph closes - protect reopened graphs from stale transaction leases

- release backend leases at request and task completion - preserve committed work and roll back unfinished writes - cover authentication and graph transaction cleanup

- retain nullable and label semantics across cleanup - refresh graph index schema identities consistently - cover vertex, edge and index operations

- coordinate scheduled services and Raft shutdown - release metadata and native database owners - cover PD lifecycle cleanup

- ship shared runtime helpers in PD and Store distributions - match native selection with component provider settings - preserve Store caller preloads across allocator paths

- drain heartbeat work before releasing Store owners - close partition and metric native resources - cover heartbeat and metric session cleanup

- close node services through the component lifecycle - avoid shutdown from locked state callbacks - preserve coordinated invalid-shard exit handling

- start the first reset after the configured interval - preserve the initial request window - cover fixed-window timing boundaries

- keep launcher regressions in the PD and Store workflow - align the PR merge fixture with the actual event SHA - use canonical ASF project references

- show prepare, configure, initialize and start in order - keep persistent paths and distributed component settings clear - use the approved illustration and a single README entry

imbajin mentioned this pull request Oct 6, 2026

feat: enable ToplingDB for Server/PD/Store (2/4) hugegraph/hugegraph#266

Closed

6 of 11 tasks

imbajin mentioned this pull request Oct 6, 2026

chore: align review rules for code and Markdown #3274

Closed

3 of 9 tasks

imbajin force-pushed the task/topling-core-20261005 branch from 40994eb to 7ee8766 Compare October 6, 2026 10:26

MrJs133 previously approved these changes Oct 6, 2026

View reviewed changes

imbajin dismissed MrJs133’s stale review via dd9399d October 6, 2026 13:50

imbajin force-pushed the task/topling-core-20261005 branch from 7ee8766 to dd9399d Compare October 6, 2026 13:50

imbajin mentioned this pull request Oct 6, 2026

[1.8.0] Release TODOs and cross-repository integration checklist #3242

Open

51 of 96 tasks

bitflicker64 reviewed Oct 7, 2026

View reviewed changes

Comment thread ...rver/hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphIndexTransaction.java Outdated

imbajin mentioned this pull request Oct 7, 2026

docs: explain ToplingDB runtime setup (2/4) apache/hugegraph-doc#510

Open

imbajin force-pushed the task/topling-core-20261005 branch from bf666f8 to 51453a2 Compare October 8, 2026 08:59

imbajin added 15 commits October 8, 2026 20:31

fix: align secure runtime dependencies

ceaf793

- align Netty, gRPC and protobuf runtime versions - preserve module dependency and test configuration - include packaged dependencies and matching licenses

feat(commons): select the RocksDB runtime

3595831

- add explicit provider selection and compatibility checks - preserve standard RocksDB as the default - cover runtime selection and utility scanning

feat: prepare the Topling native runtime

bdde25c

- verify the JNI package and EasyMigrate capability - install readable runtime assets outside standard libraries - provide shared preload selection and native configuration

feat(server): enable explicit Topling launches

03976e6

- select one runtime for Server, init and dump commands - keep inherited configuration scoped to the selected component - cover launcher selection and upgrade contracts

fix(rocksdb): release native session owners

8294fe1

- close session and database native resources reliably - preserve commits and clear reusable write batches - cover native session disposal and reuse

fix(cache): preserve graph cache lifetime

a121400

- retain shared caches and invalidation listeners across requests - dispose graph-scoped holders when the graph closes - protect reopened graphs from stale transaction leases

fix(server): close request and graph resources

7d5cf67

- release backend leases at request and task completion - preserve committed work and roll back unfinished writes - cover authentication and graph transaction cleanup

fix(server): preserve schema and index semantics

98f3fe9

- retain nullable and label semantics across cleanup - refresh graph index schema identities consistently - cover vertex, edge and index operations

fix(pd): drain work before closing metadata

587f9ce

- coordinate scheduled services and Raft shutdown - release metadata and native database owners - cover PD lifecycle cleanup

feat: wire distributed Topling launchers

f0aa08e

- ship shared runtime helpers in PD and Store distributions - match native selection with component provider settings - preserve Store caller preloads across allocator paths

fix(store): release engine and metric resources

358218e

- drain heartbeat work before releasing Store owners - close partition and metric native resources - cover heartbeat and metric session cleanup

fix(store): coordinate node shutdown

5379e44

- close node services through the component lifecycle - avoid shutdown from locked state callbacks - preserve coordinated invalid-shard exit handling

fix(server): defer the first rate limit reset

a0d80ff

- start the first reset after the configured interval - preserve the initial request window - cover fixed-window timing boundaries

chore(ci): retain focused runtime checks

923a8b2

- keep launcher regressions in the PD and Store workflow - align the PR merge fixture with the actual event SHA - use canonical ASF project references

docs: explain the three-step Topling switch

96f99de

- show prepare, configure, initialize and start in order - keep persistent paths and distributed component settings clear - use the approved illustration and a single README entry

imbajin force-pushed the task/topling-core-20261005 branch 2 times, most recently from 22bff97 to 96f99de Compare October 8, 2026 13:31

zyxxoo approved these changes Oct 8, 2026

View reviewed changes

imbajin merged commit 134300a into apache:master Oct 8, 2026
36 checks passed

imbajin deleted the task/topling-core-20261005 branch October 8, 2026 16:06

This was referenced Oct 8, 2026

feat: integrate ToplingDB (split source only) hugegraph/hugegraph#264

Closed

fix: release query resources and drain Store RPCs hugegraph/hugegraph#261

Open

fix: release query resources and drain Store RPCs #3294

Open

Navigation Menu

Repository navigation

feat: add explicit ToplingDB runtime selection - #3275

feat: add explicit ToplingDB runtime selection#3275
imbajin merged 15 commits into
apache:masterfrom
hugegraph:task/topling-core-20261005

imbajin commented Oct 6, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

bitflicker64 left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

imbajin commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of the PR

How to switch

Main Changes

Verifying these changes

Does this PR potentially affect the following parts?

Documentation Status

Uh oh!

codecov Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bitflicker64 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

imbajin commented Oct 6, 2026 •

edited

Loading

codecov Bot commented Oct 6, 2026 •

edited

Loading